Using mixed methods for evaluating the effect of a quality improvement collaborative for management of sleep problems presenting to primary care

Context This improvement project was set in Lincolnshire, a large rural county in the East Midlands with high prescribing rates of hypnotic drugs compared with the rest of England. Eight general practices volunteered to participate in a Quality Improvement Collaborative (QIC) designed to improve management of sleep problems in patients presenting to primary care. Problem Sleep problems are common affecting around 40% of adults in the UK. Insomnia has considerable resource implications in terms of disability, impaired quality of life and health service utilisation. Up to half of individuals with Insomnia seek help from primary care and hypnotic drugs are often inappropriately prescribed for long term use. Non-pharmacological treatment measures are rarely implemented in practice despite guidance supporting their use. A lack of training as well as limited availability of resources for effective sleep assessment and treatment in primary care are possible explanations for this. It is clear that there is considerable scope for improving management of sleep problems in general practice Assessment of problem and analysis of its causes We used a Quality Improvement Collaborative to introduce practitioners to sleep assessment tools including the Insomnia Severity Index (ISI), Pittsburgh Sleep Quality Index (PSQI) and Sleep Diaries and non-pharmacological interventions such as Cognitive Behavioural Therapy for Insomnia (CBTi). Practitioners from participating practices were asked to begin using these where appropriate within their day to day practice. Strategy for change The project team met bi-monthly with practice teams to share learning. We used adult learning techniques to promote rapid experimentation (Plan, Do, Study, Act) cycles, process redesign and monthly feedback of prescribing rates and costs of hypnotic drugs using statistical control charts. Data were collected from the collaborative meetings to understand the facilitators, barriers and changes that practices were making as a result of the Quality Improvement Collaborative (QIC). Measure of improvement Qualitative data were collected via audio recordings of practice and collaborative meetings with practitioners and practice staff. This data was then transcribed verbatim. Thematic analysis was carried out supported by computer software MaxQDA using a framework method. Nine themes emerged which were then reviewed by five members of the evaluation steering group to assess inter-rater reliability of the themes. We used statistical process control charts and an interrupted time series design to analyse prescribing data for the two year period preceding the establishment of the collaborative and for the six months of its operation. Effects of changes There was a significant reduction in hypnotic prescribing of benzodiazepines and Z drugs in the practices over the six months of the project and this improvement has been sustained since the initiative. Nine themes emerged from the qualitative data: - Engagement of staff: Most practitioners showed enthusiasm to incorporate changes in their practice and encouraged other members of the practice to become involved by demonstrating use of the tools and reminders during meetings “It’s brought up at every practice meeting and so it’s always fresh in people minds. It’s not something that’s then forgotten.” Practitioner views of the tools: Practitioners tried the tools and techniques and overall seemed to favour the Sleep diary and Insomnia Severity Index (ISI) over the Pittsburgh Sleep Quality Index (PSQI) “Generally we found that the ISI was easy to complete, score and interpret and can be used in general practice” Practitioner preconceptions: Practitioners came with preconceptions about the feasibility of sleep tools and techniques. Patients’ age and intellect were factors that practitioners thought might affect whether tools were completed correctly or at all. Needs & educational needs of patients & staff: Before this project hypnotics had been seen as the solution to most sleep problems by both patients and practitioners. “When people come in it was so easy to give them a prescription” "As GPs we’re overly limited and actually to have a slightly more sophisticated response would actually be better for us but also for the patient”. Barriers to implementing tools & techniques: This related to systems (of care) practitioners and patients Systems: “Once the psychiatrist says you should have this, it is really hard as a GP to go against it because you know they say the psychiatrist has asked me to take this.” Practitioner: “We come down to the cognitive behaviour therapy approach; it’s a bit thin on my part, we’ve not got great skills in that”. Patient: “I think the key is also definitely how to communicate it…the minute you start even trying to approach the subject that the tablets are not really very good and what about thinking about alternative ways, they will kind of glare very rudely and be like I have been there before doc[tor]. So you have got to kind of approach it in a kind of a fresh way to make them thing they are trying something new. You have got to be a salesman’. Changes initiated by practices: Some practices had taken other measures to try and reduce hypnotic prescribing including implementing withdrawal programmes and limiting repeat prescriptions which let to improvement is patient and practitioner experience GP-Patient treatment & expectations: Practitioners revealed what they thought patients expected and made suggestions of how consultations could be improved to meet patients’ needs and increase successful outcomes from a sleep consultation. Importance of tailored approach: Each patient with Insomnia would need to have their treatment tailored to their individual requirements therefore every consultation could potentially have very different solutions Lack of feedback from patients: Receiving feedback from patients was difficult for some practitioners when patients didn’t return for their follow-up consultation or didn’t complete and return their sleep assessment tools. This lead practitioners to feel unsure as to whether patients had read and absorbed the information provided to them Lessons learnt Qualitative methods for collecting and analysing data were invaluable in understanding the factors which helped bring about change, how change happened and the effect of the change on process of care and patient and practitioner experience Message for others Quality improvement collaboratives benefit from careful analysis using qualitative as well as quantitative methods. Further information www.restproject.org.uk Project manager: [email protected] Project lead: [email protected]

Evaluating interventions for violence prevention using linked MoJ-DfE data: a feasibility study.

ObjectiveTo assess the feasibility of using linked education and offending data (from the National Pupil Database, Department for Education and the Police National Computer, Ministry of Justice) to identify matched control groups to evaluate violence prevention interventions.ApproachWe simulated a plausible intervention (multi-systemic therapy aimed at high-risk young people living in high-risk areas) aimed at reducing the rate of serious violent offending between the ages of 15 and 18 years. We separately simulated an intervention in London and one outside London. We selected eligible individuals aged 14 years for inclusion in the intervention group, modelled the predictors of serious violent offending. then used two different matching algorithms – prognostic score matching and (coarsened) exact matching – to identify matched controls. We compared their effectiveness by measuring the observed rates of serious violence in the two groups.ResultsThe dataset we used dataset included just under 1.5 million individuals born between 1st September 1995 and 31st August 1998 with complete data. Consistent with previous research, factors associated with the risk of serious violence included deprivation, geographical region, sex, ethnicity, attainment, school absence and exclusion, being in care of the local authority or classified as in need, as well as prior offending and some school-level factors. Exact matching or coarsened exact matching was more successful than prognostic score matching at selecting suitable control groups, both within and outside London. Within London, exact matching on sex, ethnicity and any offending before age 14 gave a suitable control group; outside London it was necessary to match on a few additional characteristics in order to obtain well-balanced groups.ConclusionThe linked dataset can feasibly be used to generate suitable matched control groups to evaluate violence prevention interventions; exact matching on key characteristics is potentially the optimal solution. Its utility in practice will depend on regular data updates and having an efficient mechanism for accessing the data for such purposes

Relative age in the school year and risk of mental health problems in childhood, adolescence, and young adulthood

Purpose Relative age within the school year (‘relative age’) is associated with increased rates of symptoms and diagnoses of mental health disorders, including ADHD. We aimed to investigate how relative age influences mental health and behaviour before, during and after school (age range: 4–25 years). Method We used a regression discontinuity design to examine the effect of relative age on risk of mental health problems using data from a large UK population-based cohort (Avon Longitudinal Study of Parents and Children (ALSPAC); N = 14,643). We compared risk of mental health problems between ages 4 and 25 years using the parent-rated Strengths and Difficulties Questionnaire (SDQ), and depression using self-rated and parent-rated Short Mood and Feelings Questionnaire (SMFQ) by relative age. Results The youngest children in the school year have greater parent-rated risk of mental health problems, measured using parent-rated SDQ total difficulties scores. We found no evidence of differences before school entry [estimated standardised mean difference (SMD) between those born on 31 August and 1 September: .02 (−.05, .08)]. We found that estimates of effect size for a 1-year difference in relative age were greatest at 11 years [SMD: .22 (.15, .29)], but attenuated to the null at 25 years [SMD: −.02 (−.11, .07)]. We did not find consistent evidence of differences in self-rated and parent-rated depression by relative age. Conclusions Younger relative age is associated with poorer parent-rated general mental health, but not symptoms of depression

Using Instruments for Selection to Adjust for Selection Bias in Mendelian Randomization

Selection bias is a common concern in epidemiologic studies. In the literature, selection bias is often viewed as a missing data problem. Popular approaches to adjust for bias due to missing data, such as inverse probability weighting, rely on the assumption that data are missing at random and can yield biased results if this assumption is violated. In observational studies with outcome data missing not at random, Heckman's sample selection model can be used to adjust for bias due to missing data. In this paper, we review Heckman's method and a similar approach proposed by Tchetgen Tchetgen and Wirth (2017). We then discuss how to apply these methods to Mendelian randomization analyses using individual-level data, with missing data for either the exposure or outcome or both. We explore whether genetic variants associated with participation can be used as instruments for selection. We then describe how to obtain missingness-adjusted Wald ratio, two-stage least squares and inverse variance weighted estimates. The two methods are evaluated and compared in simulations, with results suggesting that they can both mitigate selection bias but may yield parameter estimates with large standard errors in some settings. In an illustrative real-data application, we investigate the effects of body mass index on smoking using data from the Avon Longitudinal Study of Parents and Children.Comment: Main part: 27 pages, 3 figures, 4 tables. Supplement: 20 pages, 5 figures, 10 tables. Paper currently under revie

Improved two-stage estimation to adjust for treatment switching in randomised trials:g-estimation to address time-dependent confounding

In oncology trials, control group patients often switch onto the experimental treatment during follow-up, usually after disease progression. In this case, an intention-to-treat analysis will not address the policy question of interest – that of whether the new treatment represents an effective and cost-effective use of health care resources, compared to the standard treatment. Rank preserving structural failure time models (RPSFTM), inverse probability of censoring weights (IPCW) and two-stage estimation (TSE) have often been used to adjust for switching to inform treatment reimbursement policy decisions. TSE has been applied using a simple approach (TSEsimp), assuming no time-dependent confounding between the time of disease progression and the time of switch. This is problematic if there is a delay between progression and switch. In this paper we introduce TSEgest, which uses structural nested models and g-estimation to account for time-dependent confounding, and compare it to TSEsimp, RPSFTM and IPCW. We simulated scenarios where control group patients could switch onto the experimental treatment with and without time-dependent confounding being present. We varied switching proportions, treatment effects and censoring proportions. We assessed adjustment methods according to their estimation of control group restricted mean survival times that would have been observed in the absence of switching. All methods performed well in scenarios with no time-dependent confounding. TSEgest and RPSFTM continued to perform well in scenarios with time-dependent confounding, but TSEsimp resulted in substantial bias. IPCW also performed well in scenarios with time-dependent confounding, except when inverse probability weights were high in relation to the size of the group being subjected to weighting, which occurred when there was a combination of modest sample size and high switching proportions. TSEgest represents a useful addition to the collection of methods that may be used to adjust for treatment switching in trials in order to address policy-relevant questions